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(57) Image quality adjusting apparatus includes a 
display unit (15) displaying an input original image, an 
area designating unit (12) designating an arbitrary area 
of the original image displayed on display unit (15). and 
an arithmetic processing unit (14) adjusting image qual- 
ity of the input original image, based on pixel values of 
the area designated by the area designating unit (12). 
Arithmetic processing unit (14) includes an object color 
detecting unit (14a) detecting object color of the area 
designated by area designating unit (12). color adjusting 



unit (14b) for performing ccrfor adjustment of the object 
color detected by object color detecting unit (14a). and 
a luminance, saturation adjusting unit (14c) adjusting lu- 
minance and saturation of the input original image. 
Arithmetic processir^ unit (14) adjusts image quality of 
the input original image based on pixel values of the ar- 
ea designated by area designating unit (12), and there- 
fore it beconnes possible for the user to perform image 
quality adjustment to attain prescribed quality by simple 
operation. 



FIG. 5 



AREA DESIGNA TING UN Ft] 
12 



STORAGE UNIT 



[ IMAGE INPUT UN3T| - 
^11 



CM 
< 

CO 
N. 

O 
CO 
O 

o 

D. 

UJ 



ARITHMETIC 
PROCESSING UNIT 

h-14a 



— [ DISPLAY UNIT j 



OBJECT COLOR 
DETECTING UNIT 



COLOR 

ADJUSTING UNIT 



LUMINANCE-SATURATION 
ADJUSTING UNIT 



EXTERNAL STORAGE UNIT 



PiinM by Jeuvv, 75001 MRS (PfQ 



EP 0 989 739 A2 

Description 

BACKGROUND OF THE INVENTION 

Field of the Invention 



tOOOlJ The present invention relates to image processing technique used in word processors, personal computers 
work stations, personal digital assistants, copying machines, scanners. facslmUe machines, television receivers vldB<i 
ca^e recorders, video cameras, digital still cameras and so on. More specifically, the present invention relates to 
method and apparatus of Image quality adjustment for adjusting an acquired image to a state desired by an operator 
for example, for improving bnage quality of a photograph, as well as to a recording medium recording an image qualiti 
adjusting program. ** ^ ^ ' 

Description of the Background Art 

C0002I Recently, digitized Images obtained through cameras, video cassette recorders and through the Internet come 
to be widely used in various fields. Sometimes these images are used as they are. and sometimes the images are 
used with Image quality adjusted. As a method of ad j ustin g I mage quarrty . aiitn ma tin artj u stment o f ima ga^juaCfa^-ota- 
photograph or an image using an image processing apparatus provided in a personal computer, a word processor or 
the like has been studied to enable reduction in necessary working time, mass processing and easy and handy image 
quarrty adjustment of a prescribed quality without requiring special skill of the operator. 

[00031 In an automata Image quality adjusting function employed In a general image processing software, a shadow 
point and a highlight point of luminance are found from luminance histogram distributkjn or the like. The histogram Is 
expanded or compressed such that the shadow point is set to a specific luminance value, for example the lowest 
possible luminance value which can be handled by the image processing apparatus, and similarly, tiie highlight point 
is set to a speaffc lumjnance value, for example, the highest possible luminance value which can be handled by the 
image processing apparatus, so that contrast is Improved and the Image quality is adjusted. 

[0004] In the Image processing method described above, the simplest manner of determining the shadow point is 
to determine a pixel having the lowest luminance among all the pixels in the image as the shadow point, and the 
simplest manner of determining the highlight point is to determine a pbcel having highest luminance among all the pixels 
in the Image as the highlight point. 

[0005] Japanese Patent No. 2527715 discloses an image processing method which is another method of adjusting 
image quality. 

[0006] In the Image processing method disclosed in Japanese Patent No. 2527715. hue value of a pixel having a 
prescribed hue among pixels of an Image is transformed to a desired hue component, so as to adjust image quality. 
[00071 The image processing techniques described above, however, have the fbUowing problems. 
[0008] First, in the method of improving oonlrast by finding a shadow point and a highlight point, it is possible to 
adjust a dark image to be lighter, or to transfomi image of which overall luminance is too high and appears white to an 
image of moderate lightness, by contrast adjustment of luminance. In an image of which color appearance is different 
from an actual object by some cause or other, however, difference or deviation In color cannot be adjusted, as contrast 
adjustment is performed with color appearance not addressed, and worse, the color deviatton may become more 
noticeable by the contrast adjustment in some cases. 

[0009] The image processing method disclosed In Japanese Patent No. 252771 5 contemplates Improved reproduc- 
tivity of skin color in an image," by adjustirig hue'value-of pixels' wlthiri specific ranges of hue and "sa'toiratican. w^ 
ranges are considered corresponding to distribution of human skin color, among pixels in an image, to be the hue value 
of pretend skin color In this image processing method, however, though pixels having hue values and saturation 
which are assumed or estimated as human skin color are adjusted to have hue values of preferable skin color, the 
color appearance of the overall Image is not adjusted, and an Image of which color Is merent from that of the actual 
object is not adjusted. 

[0010] Further. In this image processing method, the hue and saturation which are considered corresponding to 
human skin color are detemiined arbiti^arily. When Individual dWerence, variation In lighting condition and so on are 
taken into consideration, human skin can assume colors of wide range. When skins of many people are taken as 
objects, ttie hue and saturation ranges of colors which are to be addressed as skin colors would be very wide, and 
objects ottier tiian human skin that have such a color that con-esponds the color of human skin would be similarly 
adjusted to the preferable skin color. Therefore, a specific object or an area of an imaoe may highly possibly be trans- 
formed to an unnatural cotor. Conversely, when the ranges of the hue and saturation which are to be addressed as 
skin colors are limited to small ranges, It may be possible that some skin cotors are not regarded as objects of adjust- 
ment, and the object of Improving reproductivlty of skin color cannot be attained. When an image has color deviatton. 
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such possibility would be higher, and the problem that color deviation other than skin color Is left uncorrected still 
remains. 

[001 1 ] Conventionally, as Is represented by the term "White balance", color adjustment in which a portion of an image 
which should have been originally white is adjusted to be whiter has been Mridely practiced. Japanese Patent La)^ng- 
s Open No. 4-213291 . for example, discloses an electronic image input apparatus which Is a prior art example related 

to white balance adjustment on a specific image plane. 

pi012] Further, it is a common practice for color adjustment to multiply each pixel value (generally, RGB is widely 
used) of pixels constituting an image by a constant, or to apply gamma function. 

[0013] In the conventional technique, however, attention is paid only to a single color (generally, white), and colors 
10 of all pixels are adjusted using parameters determined to enhance or reftne the single color. Therefore, though the 
noted color to which attention Is paid can be well adjusted, colors of other portions are not much improved and. some- 
times, even degraded. 

[0014] When a faded photograph is read by a readir^ device such as a scanner and subjected to processing for 
recovering the original color, simple multiplication or gamma function cannot address the change of color caused by 

IS fading of photographic paper. Therefore, the object cannot be attained. 

[0015] Figs. 1A to 1C schematically represent histograms of typical pixel values when a faded photograph is read 
by a scanner. In Figs. 1 A to 1 C. the ordinate represents the number of pixels of three primary colors (R, G and B), and 
the abscissa represents pixel values representing color density of the three primary colors (R, G and B). 
[0016] As right side portions of respective histograms 101 to 103 are not aligned, it is presumed that a white portion 

20 on the image is not an ideal white but a little reddish. Further, as left side portions of respective histograms 101 to 103 
are not aligned, it is presumed that a black portion on the image is reddish black. Further, as the R histogram 101 of 
Fig. 1A is biased to the right side. It is presumed that the overall image is reddish. 

[0017] Figs. 2A to 2C schematically represent histograms resulting from conventional processing, exclusively paying 
attention to white, of the image having such histograms as shown in Figs. 1 A to 1C. As the processing was performed 
2S paying attention only to white, right side portions of respective histograms 201 to 203 are aligned, whereas the left side 
portions are not aligned, and the overall balance is not good. Referring to Figs. 2A to 2C, it Is considered that the white 
portion Is made whiter white a portion which shouki be blacic remains reddish black, and the overall image still remains 
reddish. 

[0016] If the process is performed paying attention to black, it would be the case that the portions corresponding to 
30 white are not aligned, and the overall balance Is not adjusted. Further, it Is naturally considered that even when other 

single color is used as a base for the processing, it is impossible to attain satisfactory balance of the overall image. 

Therefore, it is understood that the original cotors are hard to recover by the conventional method. 

[0019] As described above, in the conventional image quality adjusting method based on a single ootor, there is a 

problem that though the color to which attention is paid can be improved, remaining portions are left as they are or, 
35 worse, degraded. 

SUMMARY OF THE INVENTION 

[0020] An object of the present invention is to provide an image quality adjusting apparatus capable of image quality 
*o adjustment to attain prescribed quality by a simple operation. 

[0021] Another object of the present Invention is to provide an image quality adjusting apparatus capable of adjusting 
image quality by accurately detecting color deviation, insufficient contrast and the like. 

[0022] A still further object of the present invention is to provide an Image quality adjusting apparatus capable of 
performing color adjustment such that a plurality of colors to which attention is paid can be improved simultaneously. 
45 [0023] A still further object of the present invention is to provide an image quality adjusting method capable of image 
quality adjustment to attain presaibed quality by a simple operatton. 

[0024] A stUi further object of the present inventton Is to provide an image quality adjusting method capat>le of ad- 
justing image quality by accurately detecting color deviation, insufRcient contrast and the like. 
[0025] A still further object of the present invention is to provhle an image quality adjusting method capable of color 

50 adjustment to simultaneously improve a plurality of colors to which attention is paid. 

[0026] A still further object of the present inventton is to provide a medium recording an image quality adjusting 
program allowing image quality adjustment to attain a prescribed quality by a simple operation. 
[0027] A still further object of the present invention is to provkle a medium recording an Image quality adjusting 
program allowing adjustment of image quality by accurately detecting color deviatton, insufficient contirast and the like. 

55 [0028] A still further object of the present invention is to provide a medium recording an image quality adjusting 
program allowing color adjustment to simultaneously improve a plurality of colors to which attentton is paid. 
[0029] According to an aspect of the present invention, the image quality adjusting apparatus includes a display unit 
fbr displaying an input original image, an area designating unit for designating an arbitrary area of the originai Image 
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displayed on the display unit, and an arfthmette processing unit for adjusting image quaiity of the input ortainal image 
t)ased on pixel vaiues of the area designated by the area designating unit. 

[0030] The arithmetic processtng unit adjusts the Image quality of the friput oflginai image based on pixel values of 
the area designated by the area designating apparatus, and therefore it becomes possible to perform Image quality 

5 adjustment to attain a prescribed quality by a simple operation 

[0031] According to another aspect of the present invention, the image quality adjusting apparatus further Includes 
a color determining unit estimating a plurality of colors from pixel values of an input original image, a cofor transform 
function generating unit for generating a color transform function based on the colors detennlned by the color deter- 
mining unit, an image processtng unit for transforming the Input original image using the color transform function gen- 

10 crated by the color transfonm ftjnctlon generating unit, and an Image output unit for outputdng the original image trans- 
formed by the image processing unit. 

[0032] The Image processing unit transfomis the Input original image using the color transfbmt function generated 
by the color transform function generating unit, and therefore it becomes possible to perform color adjustment such 
that a plurality of colors to which attention is paid are improved simultaneousty 

15 [0033] According to a still further aspect of the present invention, the method of image quality adjustment Includes 
the steps of displaying an input original Image, designating an arbitrary area of the displayed original imago, and 
adjusting Image quality of the Input original image, based on pixel values of the designated area. 
[0034 ] As the image quality of the input original imaoe Is adjusted based on thfi p r^ai vai. .o^ of thg designated area, 
it becomes possible to perform Image quality adjustment to attain a prescribed quality by a simple operation. 

20 (0035] According to a still further aspect of the present invention, the method of image quality adjustment further 
includes the steps of estimating a plurality of colors from the pixel values of the input original image; adjusting and 
detemUning the plurality of colors, generating a color transform function based on the adjusted colors, transfonning 
the Input original Image by using the generated color transform function, and outputiing the transtonned original image. 
[0036] As the input original image is transformed by using the generated color transfonn function, it becomes possible 

» to perform color adjustment such that a plurality of colors to which attention Is paid are improved simultaneously. 

[0037] According to a still further aspect of the present Invention, the Image quality adjusting program recorded on 
a medium Includes the steps of displaying an Input original Image, designating an arbitrary area of the displayed original 
image, and adjusting image quaiity of the input original image based on pixel values of the designated araa. 
[0038] As the image quality of the input original Image is adjusted based on the pixel values of the designated area. 

30 ft beconnes possible to perform image quality adjustment to attain a prescribed quality by a simple operation. 

[0039] According to a still further aspect of the present invention, the image quality adjusting program recorded on 
the medium includes the steps of estimating a plurality of colors from pixel values of the input original image, adjusting 
and determining the plurality of colors, generating a color transform function based on the adjusted colors, transforming 
the Input original image by using the generated color transform function, and outputting the transformed original image. 

M [0040] As the input original Image Is transfonned by using the generated color transform function. It becomes possible 
to perform color adjustment such that a plurality of colors to which attention is paid are tmproved simultaneously. 
[0041] The foregoing and other objects, features, aspects and advantages of the present invention will become more 
apparent from the following detailed description of the present Invention when talcen In conjunction with the accompa- 
nying drawings. 

40 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0042] Figs. 1 A to 1 C represent exemplary htetograms of pixel values of a faded irnage. _ _ 

[0043J *'Rgsr2Ato 2C represent histograrhsof pixel values"of the image shown In Fig. 1 processed by a cbriiveritlonai 
45 image quality adjusting apparatus. 

[0044] Fig. 3 shows an example of an appearance of the image quality adjusting apparatus In accordance with the 
present invention. 

[0045] Fig. 4 is a block diagram schematically representing the configuratldn of the Image quality adjusting apparatus 
shown in Fig. 3. 

so [0046] Fig. 5 is a functional block diagram of the Image quality adjusting apparatus In accordance with a first em- 
bodiment of ttie present invention. 

[0047] Fig. 6 schemattcally represents configuration of the image quality adjusting program In accordance with the 
embodiment of the present invention. 

[0048] Figs. 7A and 7B represent images before and after image quality adjustment performed by the Image quality 

S5 adjusting apparatus in accordance with the first embodiment of the present invention. 

[0049] Rg. 8 is a flow chart representing pnscess steps of the image adjusUng In accordance with the first embodiment 
of the present invention. 

[0050] Figs. SA to 9D represent images refan^ to for describing how an area is designated, an object area is ex- 
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traded and object color is detected, In the Image quality adjusting apparatus In accordance with the first embodimerrt 
of the present invention. 

[0051] Fig. 10 Is a graphic representation of a transfbon function used for color adjustment processing. 
[0052] Fig. 11 Is a flow chart showing tn detail the coior adjusting process (86) of Fig. 8. 
[0053] Fig. 12 is a flow chart showing in detaH the luminance-saturation adjusting process <S7} of Fig. 8. 
[0054] Figs, and 1 3B are illustrations representing fonnatfon of a histogram and smoothing of tt»e histogram. 
[0055] Fig. 14 is a graphic representation of a transform functton T generated based on a histogram paramatar. 
[0056] Fig. 15 is a graphic representation of correction function C generated based on the object color. 
[0057] Fig. 16 is a graphic representation of another correction function C generated based on the object color 
[0058] Figs. 1 7A and 17B are graphic representations showing an example of correction of transfonn function T. 
10059] Figs. 1 8A and 1 8B are graphic rapresentations showing another example of conrection of the transfbmi func- 
tion T. 

[0060] Rg. 19 Is a block diagram representing funcGonal configuration of the image quality adjusting apparatus in 
accordance with a second embodiment of the present invention. 

[0061 ] Fig. 20 is a flow chart showing process steps of the Image quality adjusting apparatus in accordance with the 
second embodiment of the present invention. 

[0062] Fig. 21 is a graphic representation of transform function fr used in the image quality adjusting apparatus in 
accordance with the second embodiment of the present invention. 

[0063] Rgs. 22A to 22C represent histograms of the image having the histograms of Fig. 1A to 1C, processed by 
the image quality adjusting apparatus in accordance with the second embodiment of the present invention. 
[0064] Rgs. 23A to 23D are illustrations representing transformation utilizing HIS hexagonal color model. 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 

[0065] Fig. 3 shows an exemplary appearance of the image quality adjusting apparatus in accordance with the 
present invention. The image quality adjusting apparatus includes a computer body 51. a graphic display unit 52. a 
magnetic tape drive 53 to which a magnetic tape 54 is loaded, a keyboard 55. a mouse 56, a CD-ROM drive 57 to 
which a CD-ROM (Compact Disc-Read Onty Memory) 58 is loaded, and an Image Input unit 11 , not shown. The image 
quality adjusting program is supplied by a storage medium such as magnetic tape 54 or CD-ROM 58. Image quality 
adjusting program Is executed by computer 51 , and an operator performs image quality adjustment of the original 
Iniage by operating keyboard 55 or mouse 56 while viewing the graphic display unit 52. The image quality adjusting 
program may be supplied to the computer 51 from arwther computer, through a communication circuit. 
[0066] Fig. 4 Is a block diagram showing an exemplary configuration of the image quality adjusting apparatus in 
accordance with the present Invention. Computer 51 shown in Rg. 3 includes a CPU 60. an ROM (Read Only Memory) 
61. an RAM (Random Access Memory) 62 and a hard disc 63. CPU 60 performs processing while Inputtingybutputting 
data to and fiiom image input unit 11, graphic display unit 52, magnetic tape drive 53, keyboard 55, mouse 56, CD- 
ROM drive 57, ROM 61, RAM 62 or hard disc 63. The image quality adjusting program recorded on magnetic tape 54 
or CD-ROM 58 Is once stored In hard disc 63 from magnette tape drive 53 or CD-ROM drive 57, through CPU 60. CPU 
60 performs Image quality adjustment of the original image by loading and executing the image quality adjusting pro- 
gram from hard disc 63 to RAM 62. 

[0067] The following description specifically relates to adjustment of quality of an Image or a photograph focusing 
on skin, including one's face. The present Invention, however, Is not limited thereto, and the description is only for 
convenience. The present Invention may be applied to any photograph or image of any object, for example a landscape 
such as sky and ocean, of which processing Is desired. 

Rrst Embodiment 

[0068] Fig. 5 Is a btock diagram representirtg functional configuration of the image quality adjusting apparatus in 
accordance with the first embtodiment of the present invention. 

[0069] Refenlng to Fig. S. the Image quality adjusting apparatus for adjusting quality of an image or a photograph 
includes an Image input unit 11. an area designating unit 12, a storage unit 13, an arithmetic processing unit 14. a 
display unit 15 and an external storage unit 16. 

[0070] Image input unit 11 is for recehnng as an Input, an original Image of a photograph, for example, which is an 
object of processing. Namely, this is a unit for providing a signal (Image signal) of the object Image to the image quality 
adjusting apparatus. 

[0071] The image signal here contemplates data in the shape of a matrix for twoKJimensIonal image constituted by 
a plurality of pixels an^nged In a matrix of rows and columns, in which information values 1 and 0 are respectively 
aBotted to a tdack pixel and a white pixel, for example. More specifically, the data in the shape of a matrix represent 
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digital signals having pixeJ data of respective pixels. Each pixel data contains position data representing position of 
the pixel in the two-dimensronat image, as well as numerical value data representing display characteristic of the pixel. 
The numerical value data representing the display characteristic includes luminance data representing luminance of 
each monochromatic radiation of red» green and blue, for example. 

s I0072I As image input unit 11 . other than the above described hard disc 13. magneUc tape drive 53 and CD-ROM 
drive 67, a device for reading signals stored in storage medium such as a floppy disk, an optical disk or a video tape 
may be used. In that case, the signals read from the storage medium by Image input unit 11 are stored In storage unit 
13. Tharelbre. on the recording medium, an image signal obtained by picking up image Bght from an object by an image 
pick up device (not shown) is stored. The image pick up device includes, for example, a scanner, a digital camera and 

10 a video camera. 

[0073] An Image input unit which has an image prck up device itself and the image signals obtained by which are 
directly stored In the storage unit 13 of the image quality adjusting apparatus without using any recording medium, 
may be used as image input unit 11. 

£0074] The image signal may be generated by an apparatus and method other than the image pick up device de- 
« scribed above. For example, the Image signal may be of an image generated by a third party, other than the operator 
of the Image quaUty adjusting apparatus, which is distributed by a recording medium such as Ct>«OM. The image 
signals may be obtained beforehand, or may be obtained Immediately preceding the image processing operation by 

the image quality adjusting apparatus, which will be described later. 

[007S] Area designating unit 12 is for the user to designate an area of an Image. Other than mouse 56 described 
M abova, a pointing device such as a track bail or a pen may be suitably used. The operator, using the pointing device 
such as a mouse as the area designating unit 12. designates a desired area, whDe viewing the original image input 
through the image input unit 11 and displayed on display unit 15, which wUI be described later. At this time, the desig- 
nated area Is stored In storage unit 13 and reflected on display unit IS. namely, displayed as a closed curve, for example. 
The area thus obtained Is regarded as the designated area. 
25 [0076] Storage unit 13 is for storing the designated area and Image signals of the original image, and used as a 
temporary storage area necessary for image quality adjusting process and as a program storing area for various 
processings. RAM 62 mentioned above Is the storage unit here. 

[0077] External storage unit 1 6 represents, other than the magnetic tape drive 53 and CO-ROM drive 57. a storage 
unit capable of reading information on a recording medium such as disks including DVD (Digital Versatile Disc), a video 
30 tape and the like. When riecessaiy information is not in storage unit 13. the informatkin recorded on the recording 
medium, for example, the processing program or data may be transfen-ed to storage unit 1 3 through external storage 
unit 16. 

[0078] Display unit IS may be a cathode ray tube (CRT) 52. a liquid crystal display unit or the like, for displaying, on 
a screen, the image data stored in storage unit 13 or the designated area. On the screen, the input original image is 
35 displayed, and the area Input by the area designating unit 12 by the operator is displayed in the form of a closed curve, 
for example, on the original Image. The display unit 15 also displays an output image generated by the image quality 
adjusting operation by the arithmetic processing unit 14. which will be described later. 

[0079] Arithmetic processing unit 14 is for performing the main process of the present invention, and CPU 60 men- 
tioned above is the arithmetic processing unit Arithmetic processing unit 14 Includes an object color detecting unit 14a 
40 for detecting color of an object to be processed, a cokjr adjusting unit 14b for adjusting the color, and a luminance 
adjusting unit 14c for adjusting luminance, and performs the image quality adjusting process. 

[0080] Arithmetic processing unit 14 performs the image quality adjusting process by using programs for respective 
processes stored in storage unit 13. 

■[0081] " Fig. 6 i-epresents an exemplary program for Image quality' adjustment. ' ** " 

^5 [0082] As can be seen from Fig. 6. the image quality adjusting program Includes an object color detecting program 
2 la, a color adjusting program 21b and luminance- saturation adjusting program 21c. 

(0083] These programs for processing by arithmetic processing unit 14 may be stored in advartce in storage unit 1 3. 
or may be stored on a recording ntedium 21 such as a floppy disk, an optical disk including CD-ROM or DVD, or a 
video tape, and read by external storage unit 16. All Ihe processing programs need not be stored on recording medium 
so 21, and at least one processing program may be recorded, among the aforementioned three programs for processing. 
In that case, the processing program not recorded on the recoixJing medium may be stored in advance in storage unit 
13. In this manner, capacity of storage unit 1 3 which is relatively expensive when compared with external storage unit 
16 can be saved, and hence necessary cost can be reduced. 

[0084] Image quality adjustment executed by the image quality adjusting apparatus in accordance with the first em- 
55 bodiment of the present Invention will be described in the following. Here, desgiptlon will be given assuming that the 
area designated by the operator is a human face. 

[0085] Rgs. 7A and 7B represent the Image quality adjusting operation effected by the present image quality adjusting 
apparatus, and Rg.8 Is a flow chart representing the fkiw of iniage quality adjustment In the image quality adjusting 
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apparatus of the present invention. The operation will be described, additionally referring to Figs. 9 to 23. 
[0086] Specific displays given in the image quality adjustir^ process performed by the innage quality adjusting sp' 
paratus are as follows. Fig. 7A represents an original image 3 la, and an operator designates an area 31b, designating 
the face of the object person. Thereafter, image quality adjusting process is performed, and the resulting image 32 is 
output as shown in Fig. 7B. More specifically, image quality adjusting process is performed based on the information 
of color, luminance or the like of designated area 31b. 

[0087] The image quality adjusting process by the image quality adjusting apparatus is performed through the proc- 
ess steps shown in Fig. 8. 

[0088] First, an original image is input (S 1). More specifically, image input unit 11 stores an object original image 

31a in storage unit 13. 

[0089] Thereafter, the original image is displayed (S2). Namely, display unit 15 displays the original image 31a stored 
in storage unit 13 on the screen. Thereafter, area designation is performed (S3). The operator, while viewing the original 
image 3ia displayed on display unit 15, designates an area to be processed, using area designating unit 12. 
[0090] Thus, designated area 31 b is displayed on original image 31a. as shown in Fig. 7A. The designated area is 
stored as area information, for example, as a series of coordinates or two-dimensional image data (mask image), in 
storage unit 13. 

[0091) After the information of the designated area is stored in storage unit 13. the process of the next step S4 starts. 
[0092] Then, the Object area is extracted (S4). 

[0093] Extraction of the object area will be described in detail with reference to Figs. 9A to 9D. 
[0094] Figs. 9A to 9D are schematic views related to designation of the area, extraction of the object area and de- 
tection of object color, in the flow chart of Fig. 8. An image 41a of Fig. 9A represents a portion of original image 31a 
of Fig. 7A, and an area 41b corresponds to designated area 31b. In the following, image 41a. which is a part of original 
image 31a. will be considered the original image, and area 41b will be considered as designated area 31b. 
[0095] On original image 41a of Fig. 7A stored in storage unit 13. the operator designates area 41b, using the area 
designating unit 12 described atx>ve. A mask image 42 representing the designated area of Fig. 7B is stored in storage 
unit 13. Object color detecting unit 14a of arithmetic unit 14 calculates pixel information on the original image corre- 
sponding to the designated area, that is. corresponding to white pixels of the mask Image, with reference to mask 
image 42. 

[0096] Here, average and valiance of pixel values of the original image in the designated area, for example, are used 
as pixel information. Based on the pixel information, the original image 4 1 a is divided into an object area corresponding 
to the area designated by the user, that is. area 43b represented by white pixels in image 43a of Fig. 9C. and other 
area, that is. area 43c (gray area) represented by black pixels in image 43a. The area 43b will be referred to as the 
object area, and image 43a will be referred to as object area image. In this manner, the object area is extracted. 
[0097] There are numerous methods of extracting the object area from pixel information. As an example, an object 
area extracting method will be described In which pixel information includes average and variance of pixel values of 
the original image in the designated and in which hue value represents the pixel value. 

' [0098] In the following, the terms hue. saturation and luminance will be defined as follows. When numerical value 
data representing display characteristics of respective pixels are available, for example, when luminance of monochro- 
matic radiations of red, green and blue are available, for example, based on the luminance of the monochromatic 
radiations, hue. saturation and luminance (intensity) transformed by HIS hexagonal color model popularly used in the 
field of image processing (hereinafter referred to as HIS) are used. Hue. saturation and luminance may be defined in 
accordance with different methods. For example, luminance may be an average value of luminance values of respective 
monochromatic radiations. 

[0099] The HIS hexagonal color model will be described briefly in the following. 

[0100] Assume a cube which is in contact with three axes in an RGB orthogonal coordinate system. Here. R, G and 
B and complementary colors thereof, that is. C (cyan), M (magenta). Y (yellow) satisfy such positional relation as 
represented by Fig. 23A. 

[0101] Here, main diagonal axis of the RGB cube is intensity axis I. which is defined as 

i = max{R.G.B> (2.1.10) 



[0102] Parallel projection of the RGB cube on a plane orthogonally intersecting the I axis results in a regular hexagon 
such as shown in Fig. 23B. With a given point P, hue H and saturation S are defined on a plane orthogonally intersecting 
55 the I axis and passing the point P. as shown in Fig. 23C. In this manner, a hexagonal color model such as shown in 
Fig. 230 is formed between RGB and HSI. The method of mutual transform is as follows. Here, it is assumed that the 
values R, G> B. S and I are In the range of [0. 1] and H assumes the value in the range of [0. 2ji]. 
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(a) HSI transfbrm 

I is defined by the expression (2.1.10). 

i) When J = 0 

S and hi are.detennlned as follows. 



s=o I 

H - not deter uiinedj (2.1,1 1) 



II) When 1 9ft 0 

first. S is defined by the following expression. 



S = (1 - O/I 




20 


Thereafter, values r. g and 


wnerei i = min{K,G,B) 
(min ts a function prowdir^ minimum value) (2.1.12) 
b are calculated In accordance with the following equations. 




25 










30 


r = (I-R)/(I-i)' 
g = (I-G)/(I-i) 
b = (I-B)/(I-i)^ 




(2.1.13) 






Finally. H is defined in the following mannen 






95 












when R = I. H = j(b-g) 






40 


when G==I, H = j(2H-r-b) 


(2.1.14) 






when B = I, H=j(4+g-r) 






45 


Here, when H < 0, 2ic Is added to the value of H. 






(b) HSI Inverse transform 








SO 


i)WhenS = 0 

Regardless of the value of H. the values of R. G and B are defined as follows. 




55 




RsGaB=I (2.1.15) 






ii) When S ^ 0 

First, values h» P, Q and T are calculated in the following manner, 
when 
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when 


h 


= 0. 


R 


= 1, 


G = T. 


B = P 


when 


h 


= h 


R 


= Q. 


G = J, 


B = P 


when 


h 


= 2, 


R 


= P. 


G = I, 


B = T 


when 


h 


= 3, 


R 


= P. 


G = Q. 


B = I 


when 


h 


= 4, 


R 


= T, 


G = P, 


B = I 


when 


h 


= 5, 


R 


= 1. 


G = P. 





H = 27C, H = 0 

H*=H/(7;/3) (2.1.16) 
h = floor(H*) 

where floor (x) is a function providing maximum integer not larger than x 
P = Ix(l-S) 1 

Q-Ix{l~Sx(H-h)} I {2.1.17) 
T = Ix{l-Sx(l-H + h)}J 

J 

[01 03] Then, values R. G and B are defined in the following manner, dependent on the value of h. 



(2.1.18) 



[0104] When the average and the variance of hue values of pixets In the designated area are represented as|i|,u, 
and o^hue* respectively. probabORy density function p{x) for a given hue value x can be defined by the lollowlng expres' 
sions, assuming that distribution of hue values of pixels In the designated area is In aooordanoe with nonnnal distribution. 

p(x)~ N(ti^. o%„,) = 

p(x)=-=s e (1) 

vZtco^ 

[0105] By applying the hue value x of each pixel In the original Image 41a as an argument of p(x) of the expression 
(1 ), probability value p is calculated. When the probability value p Is not lower than a threshold value Rh set in advance, 
the pixel is regarded as a white pixel, and If the probability value p is smaller than Pth, the pixel is regarded as a black 
pixel and in this manner, the object area image 43a can be generated. Thus, object area 43b, when the operator 
designated area 41b of original image 41a. can be extracted. As a result, the area of human face skin is extracted, as 
can be seen from Fig. 9C. 

[0106] Thereafter, object color is detected (S5). 

[0107] Detection of object color will be described In detail in the foJiowing, 

[0108] For a region which Is constituted by white pixels both in the object area Image 43a shown in Fig. 9C and in 
mask image 42 generated by the designation by the operator shown In Fig. 9B, that is. for an area 44b represented 
by brightest pixels in image 44a of Fig. 9D, an average of pixel values in the corresponding original image 41a is 
calculated. Here. It is assumed that the original image 41a is a color image, and pixel values represent luminance 
values of respective monochromatic radiations <red. green and blue). Namely, calculating the average of pixel values 
means that average values mR, mG and mB of respective monochromatic radiations are calculated. 
[0109] in this manner, the object color is detected. 
[0110] Thereafter, color adjusting process is perfonfned (56). 

[0111] The color adjusting process will be described In detail in the folkiwing with reference to Figs. 1 0 and 11. Fig. 
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10 is a graph showing details of the color transfonn function for the color adjusting process, In the flow chart of Fig. 8. 
Fig. 1 lis a flow chart showing details of the color adjusting process (S6) of the flow chart of Fig. 8. In the following . it 
Is assumed that pixels In an image are represented by intensities of respective monochromatic radiations of red (R). 
green (G) and blue (B). that is, RGB, with the range of each value being 0 to 255. 

. £0112] Color transform ftinctlon.for each monochromatic radiation of red (R), green (6) and blue (B) is formed such 

that the object color detected In step S5 is adjusted to aa ideal color found In advance and that color balance of the 
overall image is satisfactory. 

{Oil 3] In the present embodiment, the following functions are generated and used. 




10 


R - fr(R) (2) 




15 


6'= fg(G) (3) 
B-= tt>(B) (4) 




20 


[0114] Here, arguments R. G and B of color transform functions of respective monochromatic radiations are pixel 
values of the image before transform, that is, of the original image, and R*. G' and B" represent pixel values after the 
present color adjusting process. 

[Oil 5] In the present embodiment, respecthre color transform functions fr. fg and fb mentioned above arB generated 
to satisfy the following four conditions. 




25 


<Condition 1> 




30 


[01161 When respective object colors detected in step S5 are represented as mR. mG and mB and pixel values of 
ideal colors having the same brightness as the object colors are represented by ImR, ImG and ImB. the following 
equations are satisfied. 

Im R * fr(mR> (5) 




35 


Im G - ^(m6) (6) 
Im B = fb(mB) (7) 




40 


[0117] The values ImR. ImG and ImB may be the values obtained by projecUng mR, mG and mB on a color space 
such as the HSl color space, maintaining ttie intensity value 1, adjusting hue IH and saturation S to those of Ideal colors 
and inverse transfbrming the resulting values to RGB space. 




45 

50 


~*Condilion2>"" ' " * " 




[0118] The overall image Is analyzed, and a color which Is considered to be white is detected (S 11 ). The pixel values 
of this portion are represented as whiteR, v/hiteG and whIteB. 

[0119] When we represent pbcel values of ideal white having the same brightness as the detected portion by IwhiteR, 
(whiteG and IwhlteB. the following equations are satisfied. 

IwhiteR > fr(whiteR) (8) 




5S 


(WhiteG = ^(vtrhiteG) (9) 






IwhiteB ~ fb(whiteB) (10) 
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[01 20] There are various methods available for detecting the color (whrteR. whlteG and whiteB) which is considered 
white. In the present embodiment average values of pixel values of those pixels which have colors close to white 
among the pixels of the original image are used as whiteR» whiteG and whIteB. 

t0121I The values IwhiteR. IwhiteG and IwhiteB may be the values obtained by projecting values whIteR. whiteG 
5 and whites to the HIS color spaoe. for example, maintaining intensity I and adjusting hue H and saturation S to those 
values of ideal white (H = arbitrary value. S » 0), and by inverse transforming the resulting values to RGB space. 

<Condiaon 3> 

10 [0122] The following equations are satisfied. 



0 « frtO) 



0 = ^(0) 



(11) 



(12) 



0»fb(0) (13) 

<Condition 4> 

[0123] The following equations are satisfied. 

265 = 1r<255) (i4) 



255»fg(255) (15) 

30 

255 = fb(255) (I6) 

[0124] Thereafter, color transform function based on white and the object color is generated (S12). 
35 [0125] There can be indefinite number of color transfomn functions simultaneously satisfying the above described 
four conditions. In the present embodtment. one obtained by connecting, by lines, four points ((0, 0). (nrtX. ImX), (whiteX, 
IwhiteX). (255, 255)) (where X is any of red (R). green (G) and blue (B) components) provided in accordance with the 
above described conditions is used. 

[01 26] Fig. 1 0 shows the four points in accordance vrith the above described conditions ((0. 0). {mR. ImR), (whiteR. 
40 IwhiteR). (255, 255)) for red (R) component connected by lines. Color transform functions for green (G) and blue (B) 
components can also be generated in the similar manner. 

[0127] For example, color transform function fr(R) of the expression (2) can be defined in the following manner Ibr 
the example of Fig. 10. 

45 

when OsR<mR: 

fr{R) = (ImR/mR) x R (2-1) 

50 

when mR ^ R<wh iteR: 
fr(R) = ((IwhiteR - ImR) / (whiteR - mR)) 
55 X (R - mR) ♦ ImR (2-2) 



when WhiteR S R S 255: 
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fr(R) = ((255 - IwhiteR) / (255 > whiteR)) 



X (R . WhiteR) 1- IwhitaR 



(2-3) 



5 



25 



30 



35 



40 



45 



SO 



[0128] Finally, using the generated color transfbim function, color adjustment is performed (SI 3). 

[0129] Thus, white Is made whiter, overall white balance is adjusted, and the object color attains to Ideal color. Since 

only color adjustment takes place here, brightness is not changed. The color adjusted image (not shown) converted 

from original image 31a by the color adjusting process Is stored In storage unit 13. 

[0130] In this manner, color adjusting process Is perfomned. 

[01311 Th© description will be returned to the flow chart of Fig. 8. 

[01 32] Next, luminance and saturation adjusting process (37) is performed. 

[0133] The luminance-saturation adjusting process will be described in detail in the following with reference to Figs. 
12 to 18. 

^[01 34] Fig. 1 2 shows the details of the luminance-saturation adjusting process (87) of the flow chart shown in Fig. 
8. representing the process for generating a transform function for adjusting luminance and saturation, utilizing irifor- 
mation of the object co(or.^\ 

[0135] First, based on the luminance and saturation of each pbcet of the input image, a luminance histogram and a 
saturation histogram are formed (S21). 

[0136] Thereafter, in order to suppress abru pt change in the histograms obtained In step S21 , smoothing is performed 
so that the histograms have smooth shape (S22). 

[0137] More specificariy. for each element, the element value of itself Is added to element values of several left and 
right elements with the element in question t>eing in the center, an average value is calculated, and the average value 
is considered the element value of the element of Interest This suppresses Influence of local noise involved in the 
image, whereby essential characteristic of the Image appears on the characteristic of the histogram. 
[0138] Fig. 13A and 13B schematically represent the process of steps 321 and 322. Fig. 13A represents the gen- 
erated histogram, and Fig. 13B represents the histogram after smoothing. 

[0139] A histogram parameter is determined (323) from the smoothed histogram obtained in step S22. 
[0140] Hera, the histogram parameter refers to a highlight value, a shadow value and a central value intermediate 
therebetween, which are represented as specific element numbers In the histogram. F^s. 13A and 13B are schematic 
representations thereof. 

[0141] Here, highlight refers to the upper limit value, an element having element number larger than which can t>e 
regarded as not contributing to representation of the image characteristic. Shadow value represents the lower limit 
value, an element having element number smaller than which can be regarded not contributing to representation of 
the image characteristic. 

[0142] As to the specific method of calculating the highlight, It can be found as an element having an element value 
exceeding a value (hereinafter referred to as a reference value) obtained by dividing total element values of the his> 
togram by a prescribed value and closest to the maximum element number. The shadow can be found as an element 
having the element value exceeding the aforementioned reference value and closest to the minimum element number. 
The prescribed value for dividing the total element values of the histogram mentioned above may be the total number 
of elements (= maximum element number - minimum element number^. 

[0143] The method of calculating highlight and stiadow elements is not limited thereto, and other methods may be 
used. 

[0144] The central value is defined as that element number histogram values (total of element values) of left and 
right sides of which are the same. 

[0145] Thereafter, based on the histogram parameter calculated in step 823. transform function T(x) for adjus&ig 
the overall histogram is generated (S24). Details of the generation of transform function T(x) will be described later. 
[0146] Thereafter, based on the object color, correction function C(x) for correcting the transform function T{x) found 
in step 324 Is generated (325). 

[0147] Details of generated of correction function C(x) will be given later. 

[0148] Finally, by multiplying the transform function T(x) calculated in step 324 by correction function C(x) calculated 
in step 325. final transform function L(x) is generated (326). 
[0149] The final transform function L(x) Is defined as follows. 



[0150] The transform function T(x), con-ection function C(x) and the final transform function L(x) described above 



U x )«C(T( x». 
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can be generated for each of the luminance and saturation. More specifically, transfonn functions of luminance and 
saturation are formed independently, in accordance with ttie above described process steps. The detailed contents of 
processing may or may not be the same. For example, in generating a transform function of saturation, when an area 
of low saturation Is desired to be not much changed, the shadow information may not be utilized among histogram 
5 parameters calculated in step S23 and the minimum element number may be fixed, so that transform of the low satu- 
ration area can be suppressed. 

[01 51] Details of the steps S24 and S25 will be described in the following. 

[01 52] Description will be given mainly focusing on generation of transform function related to luminance, represent- 
ing brightness. 

10 [0153] Further, details of the process of step S24. that is. the step of generating transform function T<x) based on 
histogram parameters (highlight, shadow, central value) calculated in step S23 will be described. 
[01 54] In the present embodiment, details of generating transfer function emphasizing the characteristic of the his- 
togram utilizing S-shaped curviiinear functbn based on logarithmic function will be described. By generating a transform 
function T(x) of brightness, that is, luminance, it becomes possible to adjust contrast of the overall image. 

'5 [0155] Fig. 14 is a schematic diagram representing how the transform function T(x) emphasizing the feature of the 
histogram by S-shaped curvilinear function is generated. 

[0156] By the application of logarithmic function between the highlight point to the central value and between the 
central value and the shadow point, transform function T(x) comes to be a function in the shape of S-shaped curve as 
a whole, with the central value being the center. 
20 [0157] First, between the highlight point to the central value, a convex curve is formed by the application of the 
following equation. 

T(x) = (maximum element • m) x 

25 

log (x/central value) / log (highlight/centrai value) m (18) 

[0158] Next, between the central value and the shadow point, concave curve Is formed by the application of the 
following equation. 

30 

T(x) = m X log ((maximum element • shadow) / 
(maximum element - x)) / log ({maximum element - shadow) / 

(maximum element - central value)) (19) 

[0159] Here, the value m used in equations (16) and (19) represents the transform destination of the central value. 
By changing the value m. the center of the S-shaped curve shown In Fig. 14 can be changed. When the value m is set 
40 as the intermediate value between the maximum and minimum elements (intermediate element of the histogram), the 
histogram values are equally distributed on the left and light sides with the center of the histogram positioned at ttie 
center in the result of transform, and hence an image of better contrast can be obtained. The value m may be arbitrarily 
changed dependent on the image to be processed. 

[0160] The transform function T(x) may be generated by same or different methods for luminance and saturation. 
45 For example, when color appearance closer to the original image is desired, a transform function based on linear 
transform, by which effect of emphasis Is relath^ely low as compared with the S-shaped curve, may be applied for 
saturation. 

[0161] Next, details of generating correction function C{x) for con-ecting the transform function calculated In step 
S24, based on the object color In step S25 will be described. 
50 [0162] In the present embodiment, the method of generating the correction function C(x) by comparing the central 
value calculated in step S23 for luminance with the object color to adaptively changing the function which will be the 
base of the correction function, will be described. 

[0163] Further, luminance value of the object color and the central value are compared. The result of comparison 
can be classified into the following three. 

55 

(1 ) luminarwe value of the object color < (central value • a) 
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(2) (central value - a) £ luminanca value of object color s (central value + o) 

(3) (central value + a) < luminance value of the object color 

[0164] Here, a is a predetemiined value, and, for example, 10% of the total numt)er of elements 
J016SJ When the result of comparison is in the range (1) or (2). correction function C(x) is generated based on the 
trapazcjdal function shown In Fig. 1 5. The trapezoidal function transforms the brightness of the object color to the ideal 
value of the object color The object color is transformed in accordance with the transform function T(x) generated in 
step S24 and thereafter re^transformed In accordance with the correction function, and therefore, the Ibllowing equation 
holds, where fc represents the brightness of the ob;ecl color and Ic represents the ideal value. 

lc»C(T(fc)) (20) 

[01 66] The trapezoidal function is generated to maintain a prescribed IncRnation with the transtbnnation point of T 
(fc) being the center as shown in Fig. 1 5. so as to maintain color gradation rin 



20 



function C(x) can be define as follows. 



CW = (if„,i„/U)xx (21.1) 

C(x) = ((lf^-if^J/(f^.f^^))X 

<*-W*^nUn (21-1) 

^max S x s 255: 
C(x) = ((255 - lf„„) / (255 . f„^)) X 

0^'^max)*^inax (21-3) 

[0167] Here, the range p in which the inclination shown in Rg. 15 is maintained (between the point (^„, If^^J and 
the point {f^^^, ff^^)) may be set to an appropriate value in advance, or appropriately changed dependent on the 
image, when information such as variance of object color in the image is available. 

[ 016ai Whe n the result of comp arison is within th e range (1) and the luminance ot the object color is lower than the 
Ideal value. It may be the case that overall Image is brighter than the luminance of the object color. When the value 
near the object color Is Increased to the ideal value, however, it is possible thai the overall Image has lower contrast 
Therefore, when luminance is to be Increased with the result of comparison being in the range of (1), an appropriate 
measure may be takers for example, an upper limit of Increase may be provided. 

[0169] When the result of comparison is within the range of (3). the correction function C(x) is generated based on 
they function shown in Fig. 16. The 7 function is represented by the following equation. 

C(x) = total element number x (x / total element number)^ (22) 

[0170] The Y type function transfonms such that the brightness of the object color attains to the ideal value of the 
nbjftr . t co l nr Mnr R s pe c i fical ly , as i n th e4Fape2oidat-fafl€tiofv^escFrt)ed-abeverthe^aloeTis-det satisfyihg- 
equatlon (20). It should be noted, however, that as y function is used in place of the trapezoidal function, brightness of 
a dariter area is increased, and therefore brightness of the overall Image Is further improved. When the result of com- 
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parison Is In the range of (3). It may be the case that the overall brightness Is daricer than the object color. Therefore, 
when the trapezoidal function is used for conecting the vldnlty of the object color as In the case where the result of 
comparison is in the range of (1) or (2). there is a possibility that vicinity of the object color only is made unnaturally 
noticeable In the overall Image. 

5 [0171J By using the correction function C(x) generated In the above described manner, the transform function T{x) 
for brightness generated in step S24 can be con-ected. to provldB ideal value for an input of the object color. 
[0172J Figs. 17A. 17B,18A and 18B are schematic representation of the final transform function L(x) provided by 
correcting transform function T(x) by correction function C(x). Rg. 17A represents transform function T(x), and Rg. 
178 represents final transform function L(x) when the state of T(fc) S ic is corrected, where fc represents brightness 

10 of the object coIot and ic represents ideal value. 

[0173] Rg. 18A represents transform function T(x). and Fig. 18B represents final transfonm function L(x) when the 
state of T(fe) > Ic is conned, where 1c represents brightness of the object color and Ic represents ideal value. 
[0174) Conection function C(x) may t>e generated by the same method or different method for saturation and lumi- 
nance. When correction based on 7 function is performed on saturation in the above described method, excessive 

IS increase Is possible in the area of low saturation. When the overall Image Is dark In general, It Is often the case that 
saturation of the low saturation area is unstable. Therefore, when the saturation of the area of low saturation is Increased 
generally, the overall output would possibty be unnatural. In such a situation, it may be desirable to use the trapezoidal 
function and not the correction function based on y function when saturation is to be increased (that is, when ideal 
saturation is higher than the saturation of the object color). Alternatively, a function satisfying the equation (22). for 

20 example, a simple Jinear function, may be used to generate conrectlon function C{x), rather than the above described 
method, as the correction function of saturation. 

[0175] Based on the color transform function generated in the above described manner, the color adjusting process 
of step S12 is performed on the original image of Fig. 7A. The resulting image obtained by fa^nsformlng the color 
adjusted Image stored in storage unit 13 rs again stored in storage unit 13 and displayed, as output image 32 shown 

2S in Fig. 7B. on display unit 1 .5. 

[0176] By the above described operation, as the specific area of the input original image is designated, it becomes 
possible to detect color deviation, insuffident contrast and the like based on the color, saturation and luminance of 
designated area of the original Image and to obtain an image of which Image quality is adjusted. 
[0177] From the foregoing, it can be understood that the present invention is applicable to a system having a unit 

30 for obtaining an Image, a unit for storing an image or a unit for displaying an imaga allowing the user to designate an 
arbitrarily area on the image by using a coordinate input unit such as a mouse, a pen or a tablet. For example, the 
present invention is applicable to word processore. personal computers, work stations, personal digital assistants, 
copying machines, scanners, facsimile machines, TV receivers, video cassette recorders, video cameras, digital still 
cameras and the like. 

35 [0178] The Image quality adjusting apparatus, the image quality adjusting method and the recording medium storing 
the image quality adjusting program in accordance with the present invention enable the following operations. 
[0179] in the Image quality adjusting apparatus stmctured as described above, an original image Input by an image 
input unit 11 is first displayed on display unit 15. On the displayed original image, an operator designates a desired 
area using an area designating unit 12, which Is a pointing davk» such as a mouse or a perK 

40 [0180] Based on the designated area, an object area which is consklered designated by the operator Is extracted, 
and the color of the object area is found. More specifically, by extracting the object area In the designated area and by 
calculating average and variance of pixel values of the original Image, the object color is detected. 
[0181] Then, a color transform function is generated so that detected object area cokir attains to an kleal color and 
that color balance of the overall Image is satisfactory, and in accordance with the color transfonn function, the original 

45 image is transformed. 

[0182] For the transformed Image, histograms of luminance and saturation are provided, transftMm funcfions are 
generated such that the luminance and saturatton of the overall image are well balanced, and at portions near the 
luminance and saturation of the detected object area color, kieal object cotor is attained. Based on the transform 
functions, the image is finally transformed. 
so [0183] In this manner, it becomes possible to realize Image quality adjustment to attain a prescribed quality in a 
simple manner, without requiring special technique or skill. 

[0184] The image quality adjusting process Is capable of adjusting contrast of luminance as well as modification of 
color devlatkm. Further, as the area Is designated, it is possible to transfonm a color of a spedRc object to an ideal 
color while performing appropriate cotor deviatton cofrocfion and contrast adjustment of portions other than the specific 
99 Object. 

[0185] By the image quality adjusting apparatus structured as described at>ove. when a portrait photograph image, 
Including a person as an object In the input original image is the object of processing. It is possible by simple operation 
to transform skin color to an kJeal ook>r and to perform appropriate color deviatk>n correctton and contrast adjustments 
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on areas other than the skin. 

C0186J As described above, the image quaSly adjusting apparatus in accordance with the first embodiment makes 
ft possible to perform Image quality adjustment to attain a prescribed quafity in a simple operation, wfthout requiring 
any high technique or skill. Further, the image adjusting process allows not only contrast adjustment of (umlnance but 
also color deviatior) correction. Further, as an area Is designated, It becomes possible to improve color repraductivity 
or a specific object while performing appropriate color deviation conrection and contrast adjustment of portions other 
than the specific object 

pi87] Further. It is possible to Improve color reproductlvity of the object area of the designated area of the original 
image, while attaining satisfactory balance of color, luminance and saturation of the overall image. Further, appropriate 
color deviation correction and contrast adjustment can be attained by a simple operation to provide ideal object color 
even at portions where luminance and saturation are dose to those of the detected object area color. 

Second Embodiment 

[0188] Fig. 19 is a block diagram representing functional structure of the image quality adjusting apparatus In ac- 
cordance with the second embodiment of the present invention. The image quality adjusting apparatus includes an 
image Input unit 601 reading a cotor image, a first color detennlning unit 602 estimating white based on the original 
image Input from image input unit 601 for adjusting and detemilning white color, a second color determining unit 602 
estimating black based on the original image input from Image input unit 601 for adjusting and determining black color, 
a third color determining unit 604 for adjusting and detennining a desired color based on the original image input from 
image input unit 601, a color transform function generating unit 605 for generating a color transfbnn function based 
on the colors adjusted by the first, second and third color detemnlning units 602. 603 and 604, an image processing 
unit 608 for transforming the original Image using the color transform function generated by color transform function 
generating unit 605. a memory 607, and an Image output unit 608 outputting an image transfomied by image processing 



£0189] Referring to Rg. 19, digital image infonnation read and input by Image input unit 601 such as a color Image 
scanner, a digital camera or the like is stored In a work area of memory 607 pixel by pixel each pixel hawng three pixel 
values of (R. G. B), and displayed on Image output unit 608 such as a CRT, a liquid crystal display unit or the like. First 
and second color deteimining units 602 and 603 estimate white and black, and determine adjusted ookirs. in accordance 
30 with a prescribed method of calculation, which wRI be described later, based on the (R. G, B) signals of the digital Image 
Information stored In memory 607. 

[0190] When there is color information to be adjusted other than white and black, a color to be adjusted is designated 
by using the third color determining unit 605, and the color after adjustment is determined. Based on the adjustment 
Infomiation of white, black and the cotor to be adjusted, color transform function Is formed at color transfonm function 

35 generating unit 605. and based on the Uius generated function, the input image is subjected to image quality adjusting 
process by image transform unit 606, and an image as a result of transform is output to image output unit 608. 
[0191] The operation and principle of the present invention vtrill be described with reforance to Fig. 20. 
[01 92] First, an original image is input through the image Input unit (S501 ). Various equipments such as a cokar image 
scanner or a digital still camera may be possible as image input unit 601. Any equipment capable of input of digital 

40 image data may be used. 

[0193] In the present invention, it Is assumed that in the digital image data, each pixel has three pixel values of (R, 
G. B) signal with each pixel value beir^g in the range of 0 to 255. Image data of ottier fomnat may be similarly addressed. 
[0194] The first cotor determining unit 602 analyzes the Input image data, and estimates or picks up a cotor which 
' Is believed to be originally White (S5Q2).'There are may be various methods for estimating the color which is considered " 

45 originally white. In the present embodiment pixels closer to white of the Input image are picked up. and an average of 
pixel values of the group of pixels is calculated. It is known that statistically, the color determined by such as average 
of pixel values is estimated to be originally an achromatic color (in this case, white}. 

[0195] Though various methods are available to detennine pixels which are ctose to white. In the present embodi- 
ment, pixels which satisfy the following relation are determined to be close to white. 



25 



unit 606. 



so 



R>180 



(23-1) 



55 



G>180 



(23-2) 



B>180 



(23-3) 
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where the pixel values are represented as (R, G. B). The pixel values of the color as a result of estimation will be 
represented as (whiteR. whiteG, whiteB). 

[0196] The first color determining unit 602 calculates pixel value of the color to be attained after adjustment of the 
color. The pixel values of the adjusted color is represented as (iwhiteR, IwhiteG. IwhiteB). Though various methods 
5 are available to calculate the pixel values. <n the present invention, the following equations are utilized. 

white! = max(whiteR. whiteG, whlteB) (24-1) 

^° IwhiteR « IwhiteG - IwhiteB = whitel (24-2) 

[0197] Similarly, the second color determining unit 603 analyzes the input image data, and estimates the color which 
is believed to be originally black (S503). Though various methods are available for estimating the color which is believed 
15 to be originally black, in the present embodiment, in the similar manner as described with reference to the estimation 
of white, pixels which are close to black are picked up from the input image, and an average of the pixel values of the 
group of pixels is calculated. Though various methods are available to determine the pixels which are close to black, 
in the similar manner as described above, in the present embodiment, a pixel satisfying the following relation Is deter- 
mined to be close to black. 

20 

R < 75 (25-1) 



G < 75 (25-2) 



B < 75 (25-3) 

30 where the pixel values are represented as (R, G. 6). The pixel values of the color as a result of estimation will be 
represented as (blackR, blackG, blackB). 

[0198] The second color determining unit €03 calculates the pixel value of the color which should be attained after 
adjustment of the color. The pixel values of the adjusted color are represented as (IblackR. I blackG, iblackB). Though 
varksus methods are available for the calculation, in the present embodiment, the folkiwing equations are used. 

35 

btacfcl min(biackR. blackG. btackB) (26- 1 ) 



IblackR - IblackG = IblackB = blackl (26-2) 

[0199] Similarly, the third color determining unit 604 obtains information of a color which should be adjusted for 
refinement other than white and black (S504). As a method of obtaining such color information, it is possible to extract 
pixels having specific hue H or saturation S from the overall image and to utilize the average of the pixel values, for 
45 example. Various methods of transform and inverse transform of the pixel values RGB to and from the hue H. saturation 
S and intensity I have been known. For example, there are following equations. 
(HSl transform) 



50 



55 



H = ArcTan ((G - B) / (2R - G - B)) 
S = ((B - R)^ + (R - G)^ * (G-B)^ /65536« 
I (R + G + B)/256 



(27-1) 
(27-2) 
(27-3) 
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(HSi inverse transform) 






- 


5 


R = 256 (1/3 ♦ 2Scos(H/6)) (28-1) 
G = 256 (l/3^cos(Hffi) + Ssin{H/2)) (28-2) 








to 


B « 256 (1 / 3 - Scos (H / 6) - Ssin (H / 2)) (28-3) 








15 


10200) When skin color is the object color, for example, certain ranges of hue H and saturation S centered on the 
hue and saturation of a skin color may be used as the aforementioned specific hue H or saturation S. The sWn color 
distribution is around the hue H of H = 0.3 and saturation S of S = 0.3, and it may be practical to pick up pixels satislyfrig 
the following ranges and to calculate an average of the pixel values. 

0<H<0.6 (29-1) 
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0 < S < 0.6 (29-2) 
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[02013 In the present embodiment, the user designates an area of the image using a keyboard, a mouse or the like, 
and an average value of the pixel values within the range is used. The pUel values of this color will be represented as 
(colorRi. colorGI. colorBI) where i is an integer satisfying the relation of 0 < t S n. and n represents number of colors 
to be adjusted for refinement, where 0 S n. and when n = 0. color adjustment Is performed based onfy on white and black. 
[0202] Tha third color detennining unit 604 detennines the color to be attained after adjustment of the color. For 
determination of the adjusted color, there are various methods available. For example, a method In which a user des- 
ignates an area on the input Image or on a separate image and an average value of pixel values within the area is 
used, may be possible. In the present embodiment, a color stored In advance is used as the cotor to be attained. For 
example, if the color to be adjusted is skin color, a skin color which is recognized most preferable is used. 
[0203] The skin color which is recognized most preferable mentioned above may differ In different situations. In the 
present embodiment, as an example, skin color calculated by HIS inverse transform such that hue of the skin coior is 
H = 0.3 and saturation is S = 0.3, which are the central values of distribution and Intensity 1 being the same as that of 
the color before adjustmerU (colorRt. coIorGi. colorBi) is used. The pixel value of this target color is represented as 
(IcolorRj. IcoiorGI. IcolorBi). 

[0204] Thereafter, color transform function generating unit 605 generates a color transfbmi function for cotor adjust- 
ment based on the above described process details. In the present embodiment, the following functions are generated 
and used. 

R* = fr(R) (30-1) 








45 

SO 
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G' = fg(G) (30-2) 

B'«fb(B) (30-3) 

[0205] Here, (R, G. B) represents the pixel values of the image before transfoffn. and (R*. G'. B") represents the pixel 
values after color adjustment. More specifically, the pbcei values (R", G', B*) after color adjustment are provided as a 
function of pixel values (R, G, B) before transfonn. 

[0206] In the present embodiment, the function is generated to satisfy the following five conditions. 
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<Condition 1> 
[0207] 



<Condition 2> 
[0208] 



<Condftion 3> 
[0209] 
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lwhitBR»fr(whitBR> 
IwhtteG » fg (whiteG) 
twhiteB«fb(whiteB) 

IbiackR^frfblackR) 
IbtackG B fg (blackG) 
lblackB»fb(blackB) 

looU>rRi = fr<ootorRl) 
IcolorGi » fg (ootorGI) 
loolorBI - fb (colorBi) 



<3M) 
(31-2) 
(31-3) 

(32-1) 
(32-2) 
<32-3> 

(33-1) 
(33-2) 
(33-3) 



[021 0] Here, i is an integer in the range of 0 < i S n, and n represents number of colors to be adjusted for refinement 
[0211] Further, n is in the range of 0 S n. and when n = 0, color adjustment Is performed based only on white and 
Mack, and hence this conditton Is not applied. 

<CondiUon 4> 

10212] 



O«fr(0) 
0 = «g(0) 
0 « fb <0) 



(34-1) 
(34-2) 
(34-3) 
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<Condition S> 
[0213] 



255 = fr(255) (35.1) 



255 = ^(255) (35.2) 



255 = fb(255) (35.3) 

[P214] There may be numerous transform functions satisfying the five condition above simultaneously. In the present 

IS embodiment, one obtained by connecting points satisfying the above described conditions for each function by lines 
Is used. As can be seen from Fig. 21. the abscesa represents pixel value, and therefore when transfonn condition for 
a specific pbcel value is determined and points satisfying the condition are connected by a line or a curve, the result is 

the color transform function. 

[0215] Fig. 21 is an example of the graph of Ir (x) generated in accordance with the above described conditions 

30 where n = 1 . Spedfic graphs are not provided for functions fg (x) and fb (x). as these funcUons are similar. 

[0216] Image processing unit 606 processes all pixels of the input image using the transform function generated In 
the above described manner. Thus, color adjustment by which a plurality of colors can be improved simultaneously is 
effected. A fiaded photograph, of which recovery of original colors have been diflicull by the conventional method can 
be recovered and refined by this method. * 

25 [021 7] Figs. 22A to 22C are schematic representation of histograms processed focusing on white and black. In ac- 
cordance with the method of the present Invention of the image which had the histograms of Fig. 1A to 1C. As can be 
seen from these figures, the histograms have both the right side and left side aligned. Therefore, it can be understood 
that where the color should be white is adjusted to white, and the color which should be black is adjusted to black. 
Further, the overall balance is satisfactory, and therefore it is understood that the original colors are recovered. When 

30 another color is also used, the color can be refined without affecting the overall balance. More specifically, by the 
present invention, a method and apparatus for image quality adjustment are provided by which overall color adjustment 
function is generated from a plurality of pieces of color infbmiatlon so that all colors are adjusted to desirable colors, 
and by color adjustment using the function, disadvantages of the prior method can be eliminated. 
[0218] As described above. In the image quality adjusting apparatus of the present embodiment, the color transfonn 

35 hinction is determined with reference to transform information of a plurality of pixel values. Therefore, color adjustment 
adjusting a plurality of colors noticed simultaneously for refinement, which could not be done by the prior art Is made 
possible. Further, original colore of a faded photograph can be recovered and further, even refined, which could not 
be done by the prior art. 

[0219] Although the present Invention has been described and illustrated In detail. It Is clearly understood that the 
to same is by way of illustration and example only and is not to be taken by way of limitation, the spirit and scope of the 
present Invention being limited only by the temns of the appended daims. 



Claims 

45 

1 . An image quality adjusting apparatus, comprising: 

a display unit (15) displa^ng an input original image; 

an area designating unit (12) for designating an arbitrary area of the original image displayed on said display 
so unit (15): and 

image quality adjusting unit (14) adjusting Image quality of said input original image, based on a pixel value 
of the area designated by said area designating unit (1 2). 

2. The image quality adjusting apparatus according to claim 1 , wherein 

55 

said image quality adjusting unit (14) Includes an object color detecting unit (14a) detecting object color of the 
area designated by saM area designating unit (12). and 

a color adjusting unH (14b) performing color adjustment of the object color detected by said object color de- 
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tecting unit (14a). 

3. The image quality adjusting apparatus according to claim 2. wherein 

said object color detecting unit <14a) extracts an object area including the area designated by said area 
designating unit (12), and detects as the object color of said designated area, an average value of pixel values In 
said object area of said input original image. 

4. The image quality adjusting apparatus according to claim 2, wherein said color adjusting unit (14b) generates a 
color transform function such that the object color detected by said object color detecting unit (14a) attains to an 
ideal color and that color balance of overall said Input original image is satisfactory, and adjusts color of said input 
original image using the color transform function. 

5. The Image quality adjusting apparatus according to clabn 2, wherein 

said image quality adjusting unit (14) further includes a luminance-saturation adjusting unit (14c) adjusting 
luminance and saturation of said Input original image. 

6. The image quality adjusting apparatus according to claim 5. wherein 

said luminance-saturation adjusting unit (14c) fonns histograms of luminance and saturation of said input 
original image, generates a transform function for adjusting said histograms such that luminance and saturation 
of said input original image are satisfactory, and adjusts the luminance and saturation of said Input original image 
using the transform function. 

7. The image quality adjusting apparatus according to daim 6, wherein said luminance-saturation adjusting unit (14c) 
generates a conection function for correcting said transform function, and adjusts the luminance and saturation 
of said input original image using said transform function and said correction function. 

8. An image quality adjusting apparatus, comprising: 

a color datermining unit (602, 603. 604) estimating a plurality of colors from pixel values of an input origir^i 
image, adjusting and determining the plurality of colors; 

color transfbnn function generating unit (605) generating a color transform function, based on colors deter- 
mined by said color determining units (602, 603, 604); 

an image processing unit (606) transforming said Input original Image using the color transform ftinctlon gen- 
erated by said color transform function gener^ng unit (605); and 

an Image output unit (608) outputting an original image transformed by said image prooesang unit (606). 

9. The image quality adjusting apparatus according to dalm 8. wherein said color determining unit (602, 603, 604) 
Includes 

a first color determining unit (602) estimating white from pixel values of said input original Image and adjusting 
and determining the white color, and 

a second color determining unit (603) estimating black from pbcel values of said input original image and ad- 
justing and determining the blacic color. 

10. The Image quality adjusting apparatus according to claim 9. wherein 

said color determining unit (602, 603. 604) further includes a third color determining unit (604) adjusting and 
determining a desired color among pbtel values of said Input original image. 

11. A method of image quality adjustment, comprising the steps of displaying an Input original Image; 

designating an arbitrary area of said displayed original image; and 

adjusting image quality of said input original image, based on pixel values of said designated area. 

12. The method of Image quality adjustment according to dalm 11. wherein 

said step of adjusting said image quality includes the steps of 
detecting object color of said designated area, and 
performing color adjustment of said detected object color. 
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13. The method of image quaUly adjustment aocordUig to dalm 12. wherein 

said step of adjusting Image quafity further Includes the step of adjusting luminance and saturation of said 
Input original image. 






5 


14. A method of image quality adjustment comprising the steps of. 






10 


estimating a plurality of oolofs from pixel values of an input original Image, adjusting and detemilning the 
plurality of colors: 

forming a color transform function based on said adjusted colors; 

transforming said Input original image by using said generated color transfomn function; and 
outputting said transformed original Image. 






IS 


15. The method according to claim 14, wherein 

said step of adjusting and determining colors includes the steps of 






estimating white from pixel values of said input original image and adjusting and determining the white color, 
and 

estimating black from pixel values of said input original imaoe. and adiustino and determining th» hbirk «nior 
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16, The method of image quality adjustment according to claim 15, wherein 

said step of adjusting and determining colors further includes the step of adjusting and detemnining a desired 
color among pixel values of said input original image. 






25 


17. A medium recording an Image quality adjusting program for perfbnning the steps of a method according to any 
oneof clain^s 11 to 16. 






30 


18. Apparatus for adjusting quality of an Image ccmprislng means for determining a plurality of color values in the 
image, means for determining desired values for said color values and means for desiring a transform for transform 
said color values to said desired values, and means for applying said transfonm to at least a selected portion of 
said image. 
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19. A computer program for perfonning a method as claimed in any one of daims 11 to 16. 
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FIG. 12 
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FIG. 13A 
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FIG. 20 
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